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A  METHOD  FOR  DERIVING  PROBABILITY  DISTRIBUTIONS 
WITH  GAMMA  FUNCTIONS 


INTRODUCTION 


Of  the  many  continuous  probability  distributions  encountered  in  signal  processing,  a  good 
number  are  distinguished  by  the  fact  that  they  are  derived  from  exponential  functions  on  [0,oo), 
the  normal  (Gaussian)  distribution  being  a  notable  exception;  e.g.,  failure-rate  distributions, 
Poisson  processes,  chi-square,  gamma,  exponential,  Rayleigh,  Weibull,  and  others  involving 
exponential  functions.  The  normal  distribution  is  a  hyperbolic  function  on  (-00,4-00)  requiring 
separate  consideration.  In  signal  processing,  the  exponential  function  is  a  common  choice  to 
model  correlated  data  structures  such  as  correlated  measurement  noise  (reference  1).  Signal 
characterization  stochastic  randomness  models  also  include  exponential  functions  (see 
reference  2). 

Occasionally,  modeling  involves  functions  for  which  the  probability  density  function  (PDF) 
and  its  moments  need  to  be  derived  de  novo.  Often,  research  scientists  and  engineers  are 
confronted  with  modeling  a  random  variable  x  when  the  PDF  is  unknown.  It  may  be  known  that 
the  variable  can  reasonably  be  approximated  by  a  gamma  density.  Then,  solving  a  problem  under 
the  assumption  that  x  has  a  gamma  density  will  provide  some  insight  into  the  true  situation.  This 
suggestion  (from  reference  3,  chapter  5)  is  all  the  more  reasonable  because  many  probability 
distributions  are  related  to  the  gamma  function. 

This  report  offers  a  tutorial  approach  for  deriving  a  PDF  and  moments  for  a  certain  class  of 
[0,  c»)  exponential  functions  based  on  a  recently  derived  exponential  integral  formula.  Reference 
4  shows  how  the  present  method  extends  to  (-<»,qo).  This  work  is  an  extension  of  Naval  Undersea 
Warfare  Center  Division,  Newport,  RI,  Technical  Report  10,412  (reference  2).  The  standard  and 
elegant  approach  to  finding  moments  involving  moment-generating  functions  and  complex- 
variable  characteristic  functions  is  not  being  challenged.  Rather,  a  method  is  introduced  that  is 
based  on  applied  engineering  mathematics  and  is  both  practical  and  easier  to  implement  in  applied 
research  settings.  Freund  (reference  5,  p.  155)  makes  it  clear  that  statisticians  advocate  the  most 
straightforward  approach. 

A  general  exponential  integral  formula  has  been  derived  that  will  simplify  the  mathematics 
involved  in  deriving  PDFs  and  moments  for  a  useful  class  of  continuous  functions.  Consider  the 
following  exponential  integral  formula  of  order  n,  where  a,m,n^0,fi>0: 


0<x<oo , 


(1) 


,  m  +  I 

where  y  = - 

n 


>0,  and  r(«)  represents  the  gamma  function  that  is  discussed  below.  The 
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derivation  of  equation  (1)  Avas  presented  in  references  6  and  7.  Appendix  A  summarizes  the 
derivation.  Essentially,  equation  (1)  is  a  generalization  of  known  integral  formulas.  Equation  (1) 
is  the  basic  reduction  formula  required  to  find  PDFs  and  moments  for  a  large  class  of  univariate 
distributions.*  The  authors  refer  to  this  formula  as  the  Moi  formula.^ 

Table  1  lists  several  frequently  encountered  continuous  PDFs  taken  fi'om  Hoel  (reference 
3).  Each  of  those  densities  can  be  expressed  in  terms  of  the  a  ,m,n,  p,  /components  of  the  Moi 
formula.  To  construct  any  of  the  densities  in  table  1,  one  can  write  down  its  Moi  formula 
equivalent.  For  example,  the  exponential  density  is  0  <  x  <  oo  ,  Each  of  the  densities  in 

table  1  is  distinguished  by  the  fact  that  when  integrated  over  the  interval  0  to  oo,  each  is  equal 
to  1,  which  is  the  definition  of  a  PDF.  Later,  the  method  will  be  justified  by  deriving  the  general 
moments  formula  for  exponential  functions  like  those  in  table  1 . 

The  use  of  the  proposed  method  wll  be  justified  by  application  to  functions  with  known 
solutions  obtained  by  the  traditional  methods  of  moment-generating  or  characteristic  functions. 
The  equivalence  between  a  moment  generating  function  approach  to  finding  moments  and  the 
current  approach  will  be  demonstrated.  In  addition,  hypothetical  functions  are  introduced  and 
the  entire  method  applied  to  the  functions  to  exemplify  the  details  of  the  method.  Note  that  the 
general  case,  -oo  <  x  <  oo,  has  been  solved  (see  reference  4),  although  this  study  concentrates  on 
[0,oo)  models. 


*  See  appendix  C  for  multivariate  generalization. 

^  The  theory  of  exponentials  is  not  pursued,  but  the  Moi  formula  of  equation  (1 )  applies  to  integrals  of  the  form 

.  +eo  “  „ 

I  —  \  OCX.  e  ^  dx ,  such  that,  if  the  integrand  is  an  integrable  odd  function,  =  -h[x'j ,  then  1  =  0 


C  +CO  2 

(e.g.,  I  xe  dx  =  0 ),  and  if  the  integrand  is  an  integrable  even  function,  or  symmetrical,  about  x  =  0, 
J  -OO 

h{x)  =  h{-x),  then  7  =  f  ^  ax”'e'^'' dx  =  2  f  ^  ax”'e~^’‘dx  = 

J  -00  J  0  fljS^ 

e.g.,  [  X^e~^  '^dx  —  ^27r  (reference  4). 

J  -OO 
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Table  1.  Univariate  Densities  Based  on  Exponential  Functions 


Moi  Components 


Density* 

a 

m 

n 

P 

r 

Exponential 

1 

0 

1 

X 

1 

Gamma 

A-1 

1 

X 

A 

rc^/) 

Chi-Square 

-^-1 

1 

1 

V 

2 

2 

2 

Rayleigh 

la 

1 

2 

a 

1 

Gamma- 

d(cXj 

md-\ 

d 

cX 

m 

Poissonf 

T(m) 

Weibull 

ab 

b-\ 

b 

a 

1 

Maxwell 

-Jll 

2 

2 

1 

3 

2  2 


Note:  Densities  selected  from  reference  3. 

*  To  construct  density  functions,  substitute  the  density  components  into  the 

exponential  function  ax"'e'^'' .  Assume  that  x  is  the  random  variable  for 
each  distribution, 
t  Derived  in  reference  2. 


PROBABILITY  DENSITY  FUNCTIONS  AND  MOMENTS 


GAMMA  FUNCTION 

Because  the  integral  in  equation  (1)  involves  the  gamma  function,  a  brief  review  of  the 
properties  of  gamma  functions  will  be  given  for  reference  later  in  this  report. 

The  gamma  function  (or  Euler's  integral)  is  defined  by  the  following  improper  integral: 

CO 

r(x;a)  =  jx“''' e'^'^dx,  a  >  0  .  (2) 

0 


Equation  (2)  is  sometimes  written  Y (a)  when  x  is  understood.  Equation  (2)  arises  in  many 
applications  of  probability  theory  in  signal  processing.  The  drawn  curve  of  equation  (2)  when  x  is 
positive  is  a  U-shaped  graph  depicting  a  continuous  function.* 

The  properties  of  equation  (2)  are  summarized  as  follows  (for  proofs  see  references  3,  5,  8, 
and  9). 

1 .  r(«)  =  («- 1)!,  if « is  a  positive  integer.  (3) 

For  example,  r(4)=  3 !  =  6. 

2.  r(a  +  l)=  ar(a),  forany  a>0.  (4) 

For=xample.  =r(f +  >)  =|r(|). 

3.  r(«  + 1)  =  «!,  if  « is  a  positive  integer.  (5) 

For  example,  r(4  + 1)  =  4 !  =  24. 


4. 


{n-])\4n 


if « is  an  odd  positive  integer. 


(6) 


*  Although  r((2)  can  be  computed  for  a  <  0  (see  reference  9),  the  restriction  in  equation  (2)  is  necessary  for  reasons  that 
will  become  apparent  subsequently. 
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For  example, 


n  — ]  =  etc, 
.27  4 


The  relation  tI  —  =  is  a  famous  relation  and  has  been  proved  in  various  ways.  An 


independent  proof  based  on  the  Moi  formula  of  equation  (1)  is  given  in  appendix  B. 


For  other  cases  of  a  >  0,  numerical  integration  must  be  used  to  obtain  the  value  of  the 
gamma  function.  For  example,  =  2.67893...  must  be  calculated  by  some  quadrature 
method.  See  reference  10  for  extensive  tables  and  useful  algorithms. 


DENSITY  FUNCTIONS 

The  first  use  of  equation  (1)  occurs  when  one  desires  to  find  a  one-dimensional  PDF  for 
exponential  function  that  the  researchers  have  decided  models  their  data  satisfactorily.* 

Let  that  function  be  denoted  by 

g(x)  =  coc‘”e''^” ,  0  <  X  <  00  , 


where  a,  /?  >0,m,n  are  any  real  constants  and  T  =n  '  (/w  -i- 1)  >  0 .  Por  example,  if  the  real- 


valued  function - 


TiM) 


e  conforms  to  the  conditions  of  equation  (8), 


then 


a  =  ——,m  =  fi-\,fi  =  A,n  =  \,  andy^jn. 


>i< 

The  multivariate  generalization  of  the  method  is  contained  in  appendix  C. 


Finding  a  formula  that  determines  the  PDF  for  any  function  conforming  to  equation  (8) 
involves  transforming  (8)  so  that 

CO 

c  j  g{x)dx  =  1 ,  (9) 

0 

where  c  is  the  normalizing  constant. 

Substituting  g{x)  from  equation  (8),  equation  (9)  can  be  written  as 

CO 

c\ax”'e-^''dx^\  .  (10) 

0 

Because  the  solution  to  the  integral  in  equation  (10)  was  given  in  equation  (1),  then  c  is  found  to 
equal 


l=>c  = 


(11) 


where  y  =  >  0.  In  the  case  of  g(x),  -oo  <  jc  <  oq  c  =  -  ^  .  ,  Then,  for  any  function 

n  2^aT{r)J 

corresponding  to  equation  (8),  the  PDF^^)  is  given  by  combining  equations  (8)  and  (11): 


CO  CO 

fix)  =  c  J  gix)dx  =  J  six)dx , 


nP 


/(x)>0  (12) 

CO 

J  fix)  dx  =  \, 

0 

where X^)  will  denote  the  PDF  of  an  arbitrary  distribution.  The  Moi  density  of  equation  (12)  can 
be  verified  by  integrating;  the  result  will  be  1.  Appendix  C  treats  the  multivariate  case. 

To  give  an  example  of  equation  (12)  for  the  densities  of  table  1,  the  PDF  for  the  exponential 
function,  constructed  from  the  components  of  equation  (12)  is 
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MOMENTS 


The  moments  of  a  PDF  are  important  for  several  reasons.  The  first  moment  corresponds  to 
the  mean  of  the  distribution,  and  the  second  moment  allows  a  calculation  of  the  dispersion  or 
variance  of  the  distribution.  The  mean  and  variance  may  then  be  used  in  the  central  limit  theorem 
or  normal  approximation  formula  for  purposes  of  hypothesis  testing. 

For  an  arbitrary  moment  of  the  class  of  functions  being  considered  (where  integration  is 
restricted  to  the  interval  0  to  oo)  the yth  moment  is  defined  as; 


uu  uu 

F:(xO  =  J X^cg{x)dx  =  j x^f  {x)dx  ,7  >  0  , 


(13) 


whereX^)  is  the  assumed  PDF  of  equation  (12). 

Expressed  in  terms  of  the  normalized  Moi  density  of  equation  (12),  equation  (13)  is 


0  ) 


(14) 


£(x')  =  J 


np'  X 


m+; 


r(r) 


-e  ^"dx  . 


Equation  (14)  is  evaluated  by  the  Moi  formula  of  equation  (1).  Let  a  -  substitute  m+j 

for  the  parameter  m,  and  y  +  j/n  for  the  parameter  y  in  equation  (1)  to  obtain 


r 


;  \  -  n-Jin 


E(x^)  =  p 


7  +  ^ 


n 


-  R-j!'' 


rlr) 


m+ 7  +  1 


n 


m  +  \ 


I  n  } 


(15) 


Note  that  equation  (15)  provides  the  moments  relation  for  the  general  case,  -oo  <  x  <  co.  Equation 
(15)  is  a  closed  form  solution  for  calculating  any  moment  of  a  PDF  conforming  to  the  class  of 
exponential  functions  under  consideration.  See  appendix  C  for  the  w-variable  case. 


MEAN  AND  VARIANCE  FOR  ANY  ARBITRARY  PDF 


The  mean  and  variance  for  any  arbitrary  PDFy(^)  conforming  to  equation  (12),  can  noAV  be 
found.  The  mean,  or  first  moment,  is  given  by  settingy  to  1  in  equation  (15).  Thus,  the  mean  is 


£(x)  =  A 


-Mn 


r['^2 

V  n 
m  +  \ 


n 


(16) 


The  variance  is  defined  by 

CT==£(x=)-[£(x)f  ,  (17) 

which  requires  knowledge  of  the  second  moment.  is  computed  from  equation  (15)  for 

j  =  2;  that  is: 


r 


/w  +  3 


£(x')  =  y9-'"’ 


(18) 


For  example,  the  reader  can  verify  that  for  the  exponential  density  of  table  1,  the  mean  and 
variance  are  given  by 

ir(x)  =  l;£(x’)  =  ^;a^=^.  (19) 

In  general,  the  moments  for  all  of  the  known  densities  in  table  1  can  be  derived  by  substitution  of 
the  appropriate  parameters  into  equation  (15). 


RELATION  TO  MOMENT  GENERATING  FUNCTION 

To  contrast  the  current  approach  for  finding  moments  to  that  offered  by  the  MGF,*  consider 
the  gamma  density  of  table  1,  given  by 

/W  =  0  <  X  <  CO  .  (20) 


*  Consideration  of  complex-variable  characteristic  functions  is  omitted. 
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Now,  by  definition,  the  MGF  of  the  distribution  of  a  random  variable  x  on  [0,oo)  is  given  by  the 
following  expectation; 


M^il)  =  Ee‘ 


Ee“  =  f  e‘'f(x)dx  =  ?  '  c<^-’^dx  ,0  <  r  <  to  , 

i  !  rw 


(21) 


where  t  is  real.  Evaluating  the  integral  in  equation  (21)  directly  by  the  Moi  formula  of  equation 
(1)  (which  saves  some  effort)  results  in 


A-tJ 


(22) 


Then,  the  mean  of  the  PDF  in  equation  (20)  is  defined  to  be  the  first  derivative  of  the  MGF 
evaluated  at  r  =  0; 


£;(x)  =  M;(0)  = 


dt 


E 

X 


(23) 


The  second  and  higher  moments  are  given  by  analogous  constructions. 

The  proposed  method  for  finding  the  mean  and  variance,  or  higher  moments,  avoids  both 
the  integration  and  differentiation  needed  to  compute  the  MGF  and  moments.  For  the  gamma 
density  of  equation  (20),  the  mean  is  found  in  one  step  by  substituting  the  appropriate  parameters 
into  the  Moi  MGF  of  equation  (16).  When  this  is  done,  the  result  is 


£(x)  =  r' 


r(;/-l  +  2)  ^  p, 
r(//)  1  ’ 


(24) 


which  is  equivalent  to  equation  (23). 

This  example  suggests  that  for  special  cases  there  is  an  equivalence  between  the  MGF  and 
the  current  approach.  This  can  be  demonstrated  for  a  general  PDF  of  order  1 .  The  PDF  via  Moi 
based  on  equation  (12)  is 


/W  = 


r(w+i) 


(25) 
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and  the  mean  from  the  Moi  moment  generating  function  of  equation  (16)  is 


^(x)  =  ■ 

r(w+i) 


(26) 


Finding  the  mean  of  equation  (25)  from  the  MGF,  in  one  step,  shows 


E(x)  =  M’(0)  = 


d  Je“/(x)£& 


dt 


=  (m  +  \)p-^  , 


(27) 


Jt=0 


which  is  equivalent  to  the  result  provided  by  the  Moi  formula  method. 

The  relationship  between  the  Moi  MGF  and  the  standard  MGF  can  be  shown  for  higher 
moments.  To  show  this,  consider  the  MGF  and  from  it  the ^h  moment.  By  definition,  the /th 
moment  based  on  an  MGF  for  the  density  in  equation  (12)  is 


=  \x^fix)dx  , 

t-Q  0 


£(x')  = 


dt 


dt 


Jt=0 


which  leads  to  the  same  definition  of  the yth  moment  of  the  current  approach  given  in  equation 
(15). 

To  further  exemplify  the  moments  equations  of  this  section,  moments  for  the  densities  are 
listed  in  table  1,  and  moments  functions  for  the  selected  univariate  densities  are  listed  in  table  2. 
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Table  2.  Moments  Functions  for  Selected  Univariate  Densities 


Densit\ 


PDF* 


Moments  Functioni 


Exponential 


Gamma 


rW 


Chi-Square 


T”'  X 

_± 


r  -  2^ 


Rayleigh 


la  k  ^ 


Gamma-Poisson 


T{m) 


(cXfi 


.  r  m+ 


Weibull 


abx'’-^e- 


I 


Maxwell 


^llTux^e  2 


yn  +  j 


ylnll 


NOTE:  The  moments  in  this  table  exist  on  the  general  interval  -oo  <  ;c  <  oo  and  0  <  ;c-  <  oo  is  a 
special  case. 

*  Calculated  from  equation  (12).  t  Calculated  from  equation  (15). 


EXAMPLES  OF  THE  METHOD 


For  the  class  of  exponential  functions  under  consideration,  the  PDF  can  be  obtained  by 
substituting  coefficients  and  exponents  into  equation  (12).  From  this  function,  it  has  been  shown 
that  the  finite  statistical  moments  can  be  obtained  from  equation  (15).  In  particular,  the  mean  of 
the  distribution  can  be  found  by  using  equation  (16),  and  the  variance  is  obtained  by  using 
equation  (17)  for  the  second  moment  in  conjunction  with  the  first  moment.  In  this  section, 
several  hypothetical  functions  are  selected  for  demonstration  purposes.  The  method  is  illustrated 
by  the  following  examples. 


FIRST  EXAMPLE 

Assume  a  researcher  has  reason  to  believe  that  certain  data  conform  to  the  following 
structure: 

g(x)  =  ax^e~‘“^ ,0^0,  0<x<QO.  (28) 

The  value  for  the  parameter  is  a  known  quantity  in  practice. 

Because  it  has  been  many  years  since  this  engineering  mathematician  has  studied 
mathematical  statistics,  it  is  decided  to  turn  to  the  Moi  MGF  method  for  answers. 

The  parameters  of  the  Moi  model  here  are  a  =  ^  =  a;  m  =  n  =  2  ,  /  =  }2.  By  using 
equations  (1 1)  and  (12) ,  the  normalized  density  for  the  function  in  equation  (28)  can  be  obtained 
in  one  step.  The  normalizing  constant  is 


2{af 


(29) 
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Substituting  the  parameters  n,  /?,  and  y  into  equation  (12)  for  the  PDFy(^),  one  obtains 


f{x)  =  cg(x) , 

_  nP'x- 

r(j.)  ’  (30) 


To  find  the  statistical  moments  of  the  PDF,  the  general  moments  are  obtained  from 
equation  (15); 

f:(xO  = 


Now,  to  obtain  the  mean,  y  =  1,  and 


£(x)  = 


a~^r(2) 

yjn  12 


2 


(31) 
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The  second  moment  is  now  obtained: 


-a 


-1 


(33) 


~  la 

The  variance,  -  [-E(x)]^  is  then  readily  given  by 


1  p;r-8^ 

a\  In  J 


(34) 


SECOND  EXAMPLE 


Consider  the  exponential  function 

=  (35) 

with  parameters  a-\,  P  -  a\  m-%  n-l\  y  -){ .  In  practice,  the  value  for  the  parameter  a  will 
be  a  known  quantity.  To  summarize  the  results,  calculations  show 


c  = 


from  (11) 


from  (12) 
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E(x)  =  p- 


0+1  +  1 


from  (16) 


E{x)  =  p- 


0+2  +  1 


from  (18) 


1  r  1  1  2-7: 


2a  \^[^)  Aa 


from  (17) 


THIRD  EXAMPLE 

The  third  example  will  lead  to  the  Rayleigh  density.  The  exponential  function  of  interest  is 
g(x)  =  axe-‘^\ 

where  the  identified  parameters  are  a  =  m  =  1;  w  =  2;  /  =  1 .  Applying  the  method,  the 
results  are  summarized  in  the  following  way: 


ar{r) 


f{x)  =  cg{x)  =  2ax^e~‘^'  , 


E{x)  =  ^ 


^m  +  l  +  l 

^  1 
1/2 _ n  _  ^ 

■  2  fr 

y  n  J  \a 


from  (11) 


from  (12) 


from  (16) 


£(x")  =  + 


m  +  2  +  l 


from  (18) 


1  t:  _4-7: 
a  Aa  Aa 


from  (17) 
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FOURTH  EXAMPLE 


The  last  examples  relate  to  underwater  acoustics.  A  deterministic  acoustic  signal 
propagating  through  random  media  becomes  a  stochastic  signal.  If  the  media  is  strongly  random, 
or  the  acoustic  path  through  a  weakly  random  media  is  long,  the  acoustic  signal  reaches  a 
saturated  state.  The  probability  density  of  the  signal  intensity  in  the  saturated  state  is  exponential  ; 

p{l)  =  (!/</  >)  exp(-l/  <  I  >), 

where 

5)  =  |/(x,  t,  a)  /  /o(x,^u)f , 

<  /  >  is  the  mean  of  the  intensity, /is  the  amplitude  of  the  signal  at  the  receiver,/)  is  the  amplitude 
of  the  signal  at  the  source,  x  is  a  positional  vector  of  the  receiver,  t  is  the  time,  and  co  is  source 
frequency. 

Notice  the  Moi  parameters  a,  m,  n,  y  in  table  1  are  defined  as  l/<  />,  0,  1,  <  7  >,  1, 
respectively,  for  the  exponential  PDF. 

As  a  related  example,  the  PDF  for  the  amplitude  of  a  random  signal  that  has  achieved 
saturation  is  Rayleigh: 

p(A)  =  2A  exp^-A^j, 


where 


A  =  f{x,t,(o)lfJ^x,co). 

This  region  is  also  characterized  by  the  real  and  imaginary  parts  of  the  complex  signal  having 
Gaussian  PDF. 

Notice  that  the  Moi  parameters  a,  m,  n,  ^'in  table  1  are  defined  2A,  1,  2,  A,  1, 
respectively,  for  the  Rayleigh  PDF  (see  reference  11). 
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HYPOTHESIS  TESTING 


Once  the  moments  have  been  derived  from  a  density  function,  exact  distributional 
probabilities  can  be  calculated  and  hypotheses  may  be  tested.  For  example,  for  the  density 

/(x)  =  4j—ax^e~‘^' ,  one’ may  wish  to  know  the  probability  that  an  observed  value  is  <  2; 

V  TT 

i.e.,  P(x  <  2). .  This  involves  evaluating 


2 

P(X<2)=  f4,  -axV“'i&, 

i  V  ^ 


(39) 


which  represents  an  evaluation  of  the  distribution  function  F'(x)  [o  <  F(x)  <  ij  for  the  density 

function_/(x).  The  quantity  a  is,  of  course,  a  numerical  value  in  a  real  situation  and  the  exact 
probability  can  be  computed.  Although  equation  (39)  represents  a  one-tailed  hypothesis,  a  two- 
tailed  hypothesis  is  equally  valid;  e.g,,  P(1  <  x  <  2)  is  the  probability  that  an  observed  value  lies 
between  the  interval  of +1  to  +2.  Standard  integration  techniques  such  as  integration  by  parts  or 
Taylor-series  expansions  are  available  for  such  calculations  as  given  in  reference  10.  For 
complicated  formulas,  an  iterative  reduction  formula  has  been  calculated  in  closed  form  for  / 
being  a  positive  integer.  This  method  was  derived  by  O'Brien,  Hammel,  and  Nguyen  (references 
12  and  13). 

In  addition,  formal  hypothesis  testing  protocols  may  be  employed  once  the  moments  of  the 
PDF  are  at  hand.  The  central  limit  theorem  or  normal  approximation  formulas  are  typically  of 
interest  for  evaluation  of  simple  hypotheses.  Chebychev’s  Theorem,  which  gives  the  probability  of 
deviation  from  a  mean  regardless  of  the  distribution,  may  also  be  of  interest.  The  reference  list 
may  be  consulted  for  standard  works  on  mathematical  statistics. 

A  possible  application  of  testing  hypotheses  with  the  method  developed  in  this  report  is 
statistical  data  analysis,  in  which,  regression  analysis  techniques  are  used  to  model  a  data  set. 
Figure  1  shows  a  set  of  data  points  (o)  generated  in  some  hypothetical  time  series  that  appears  to 
conform  to  a  negative  exponential  (or  decay)  function.  The  solid  sloping-down  arc  labeled  g[x) 
is  assumed  to  be  an  optimum  least-squares  solution  derived  for  the  discrete  time  series  data. 


g(x) 


Figure  1.  Hypothetical  Data  Conforming  to  Decay  Function 
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The  function  g(x)  is  obtained  in  the  standard  manner  for  exponential  functions.  First  take  the 
(natural)  logarithm  of  the  modeling  basis  (exponential)  function  y  -  g(x)  =  ; 


log[g(x)]  =  log  ax”e 


-A" 


loga  +  Twlogx-ySc". 


Because  the  term  ySc"  is  nonlinearizable,  a  nonlinear  approach  must  be  taken.  This  approach 
consists  of  performing  the  regression  analysis  on  g[x) ,  with  the  nonlinear  parameter  n  set  to  a 

specific  integer  value,  «  =  1,  2,  3,  etc.  Each  [x.j]  observation  in  the  sample  is  indexed  with  the 
subscript  i,  where  /  run  from  1  to  p.  Then  construct  the  least-squares  criterion  for  a  class  of 
solutions: 


fXa,m,p,n)  =  ^[log>^.  -loga  -mlogx.  +  A”]'  ^  min. 

2=1 

The  nonlinearity  of  the  problem  is  apparant  if  one  tries  to  take  the  partial  derivative  of 
fJa,m^P,n) ,  with  respect  to  n. 


-\o%a-m\Q%x. 


dn 


To  circumvent  this  problem,  a  class  of  linear  regression  solutions  is  generated  by  fixing  the 
value  for  n.  That  is,  the  first  solution  set  is  called  /, ,l)  because  it  is  assumed  that  the 
parameter  «  =  1  in  the  least-squares  regression  solution.  Next,  we  find  the  least-squares  solution 
set  member  by  setting  /?  =  2.  In  this  manner,  a  set  of  solutions  is  generated  by 

systematically  varying  the  value  for  the  parameter  n.  Of  the  entire  set  of  least-square  solutions, 
the  preferred  solution  set,  say  with  n  =  k,\s  called  provides 


the  smallest  residual  error  (root-mean-square  value).  The  solution  g-(x)  org^(x)  =  c6ce~^"  is  then 

obtained  by  taking  antilog  values  of  the  parameter  estimate  set  a  ,  the  empirical  least- 

squares  solution  to  the  data  generated  in  the  time  series.  From  g,^{x) ,  one  could  find  the 
quantities  earlier  derived  (the  probablity  density  function  and  the  moments). 


18 


SUMMARY 


To  summarize  the  results  of  this  work,  figure  2  shows  the  method  steps  for  one-dimensional 
continuous  distributions.  To  give  a  verbal  description  of  the  method,  if  an  exponential  function  is 
of  the  form 


g(x)  =  cxx^'e  .  0  <  X  <  00 , 


(40) 


where  a,  m,  n^O,  p>0,  and  y  =  n  '  + 1)  >  0,  then,  to  form  a  PDF,  compute  a  normalizing 

constant  c 


c  = 


aT{y)  ’ 


(41) 


which  provides  the  PDF, 

/(x)  =  cg(x)  =  ^^^e'^”, 

and  the yth  statistical  moment, 

r 


(42) 


J\-  O-J/K 


V  n 

W 


m+j  +  \ 


\  n 


[mVi 


\  n  ) 


(43) 


such  that  the  mean  (first  moment)  is 


E{x)  =  p 


-Vn  ^  n 


m  +  2 


m  +  r\  ’ 


n 


(44) 


and  the  variance  is 


<7^  =  £(x=)-[£(x)]\ 

where  the  second  moment  is  computed  fi-om 


(45) 
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£(x')  =  A 


Y\ 

-Vn  ^  n  . 

m+\ 


Figure  2.  Summary  of  Moi  Method  Steps 
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The  authors  of  this  report  have  derived  a  general  solution  on  (-qo  <  x  <  oo).  The 

equivalence  between  the  current  approach  and  moment-generating  functions  was  shown  generally. 
Other  information  was  provided  on  testing  statistical  hypotheses  from  the  central  limit  theorem 
and  approximation  formulas. 

Finally,  a  generalization  of  the  method  of  this  report  to  multivariate  distributions  was  given 
in  a  separate  appendix.  There  it  was  shown  that  the  joint  PDF  for  n  independent  random  variables 
is  the  product  of  individual  densities  of  the  form  given  in  equation  (42): 

/(xi,X2,...,x„)  =  n/(xJ  (47) 

i=i 


and  the ^h  multivariate  moments  are  obtained  by  forming  the  product  of  individual  moments  for 
each  random  variable 


£(y')=n£(x'). 

where  y  ^  X.  . 


(48) 


CONCLUSIONS 


This  report  has  presented  a  simple  substitution  method  for  finding  a  probability  density 
function  (PDF)  and  its  statistical  moments  for  an  arbitrary  exponential  function  of  the  form 

gix)  =  coc"'e~^" ,  0<x<oo, 

where  a,  P>  0,m,n  are  any  real  constants  in  one-dimensional  distributions  and 
in  the  hyper  plane.  Such  distributions  arise  frequently  in  signal  processing,  where  x  may  represent 
time  starting  at  f  =  0  and  g  may  represent  some  error  or  disturbance  structure  conforming  to  a 
decay  function. 

The  general  case,  when  -oo  <  x  <  c»,  has  subsequently  been  solved  (see  reference  4),  the 
primary  use  of  the  method  is  to  test  statistically,  hypotheses  about  the  behavior  of  such  functional 
forms  once  empirical  least-squares  methods  have  determined  an  applicable  model  derived  from 
actual  measurements  (such  as  regression  analyses  of  the  time  series  in  the  ocean  environment). 
That  is  to  say,  once  the  parameters  a,  p,  m,  and  n  are  determined  for  a  set  of  data  measurements, 
the  PDF-based  mean  and  variance  are  determinable,  and  simple  binary  hypotheses  may  be  tested, 
such  as  "the  error  term  is  essentially  zero."  Such  tests  are  valid  because  they  are  based  on  the 
central  limit  theorem  for  sufficiently  large  n  or  Chebychev's  theorem,  both  of  which  may  be 
employed  regardless  of  the  underlying  distribution. 


mil 
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APPENDIX  A 

DERIVATION  OF  THE  MOI  FORMULA 


The  improper  exponential  integral  to  be  derived  is 

CO 

/  =  Jccc^e"^  ,  0<  X <  00 ,  (A-1) 

0 

where  or,  m,  y9>0,  y  =rf\m  +  \)>0. 


The  integral  I  clearly  exists  since  the  integrand  is  a  continuous  function  which  is  bounded  by 
an  integrable  function  (reference  9). 

To  evaluate  I,  set 

I 

s  =  ySr” ,  so  that  x  =  =  n^x^'^dx  ,  (A-2) 


and  substituting  into  I 

CO 

/  =  a(«y0)-’Jx'""'-V^£fe  , 

0 

so  that  substituting  y  gives 

00 

I  =  a{np^)-^\s^-^e-^ds  , 

0 

where  the  integral  is  seen  to  be  the  gamma  function  r(y),  so  that 
/  =  (a'‘nA')''rO'), 


which  completes  the  derivation. 


(A-3) 


(A-4) 


(A-5) 
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APPENDIX  B 


INDEPENDENT  PROOF  THAT 


In  this  appendix,  an  independent  proof  is  given  for  the  well-known  relation  between  the 
gamma  function  and  n,  viz.,  .  To  do  this,  the  derivation  of  the  Moi  formula  is  again 


outlined  to  make  this  appendix  self-contained,  which  allows  the  authors  to  relate  their  version  to 
standard  approaches. 


The  gamma  function  is  given  as 

CO 

r(x;a)  =  |x““’e"''<ic,  a>0  ,  (B-1) 

0 


so  that  for  a  =  — ,  rf— 1  =  The  proof  of  this  statement  has  been  given  in  various  places. 


The  calculus  text  by  Bers  (reference  14,  pp.  402-403)  gives  one  approach.  Since 


00  00 

j  =  j  2  ,  (B-2) 

-00  0 

by  transformation  to  a  double  integral  in  polar  coordinates,  then  by  substituting  in  equation  (B-1) 
the  quantities  x  =  y^,  dx-  Ix^'^dy, 


00  00 

=  =  =  ,  (B-3) 

0  0 

The  proof  by  Carr  (reference  15,  theorem  2314)  proceeds  along  similar  lines.  Other 
approaches  exist  (e.g.,  reference  9). 


NEW  PROOF 

Our  version  involves  first  a  generalization  of  an  elementary  exponential  integral  found  in 
standard  handbooks.  Consider  the  following  integral: 


B-1 


(B-4) 


CO 

jax"'e~^'’dx,  0<x<oo  , 

0 

where  (a  7!=  0,  >  0,  m,  n  ^  0)  are  real-valued  constants.  To  evaluate  equation  (B-4),  make  the 
change  of  variable  by  setting  s  =  I3x”,  so  that  x  =  ,  ds  -  nl3x''~^dx,  and  letting 

/n  +  1  „ 

Y  = - >  0 ,  to  get 

n 


a{np^^  "ds,  5>0  .  (B-5) 

0 


The  integral  in  equation  (B-5)  is  recognized  to  be  a  form  of  the  gamma  function,  in  (B-1): 


I ^e~"ds  =  r {y),  y>0. 

0 


Thus, 


9.1 

I 


ax’”e-^"dx  =  a^^  , 


(B-6) 


which  is  the  Moi  formula  introduced  earlier. 


With  this  integral  formula,  consider  the  following 


Je  dx. 


which  can  be  evaluated  by  equation  (B-6)  and  simplified  to 


n) 


(B-7) 


With  this  structure,  two  of  the  known  approaches  to  the  proof  can  be  connected.  With  «  =  2, 
then 


ip''^^e-^^dx  =  T 
0 


(B-8) 


B-2 


so  that  if  P=\,  then  from  equation  (B-2) 


(B-9) 


To  approach  the  proof  another  way,  let  P  =  —  in  equation  (B-8),  giving 


(B-10) 


Now,  textbooks  on  calculus  and  probability  theory  prove  (via  polar  coordinate  transformation) 
that  the  integral  in  equation  (B-10)  is  related  to  the  probability  integral 


(B-11) 


or 


f  _xV  (•  _*y 

J  e  =  J  2e  . 


(B-12) 


Evaluating  equation  (B-1 1)  now  by  the  Moi  formula  of  equation  (B-6)  produces 


(B-13) 


and  so  equating  (B-12)  and  (B-13)  produces  the  desired  result: 


The  proof  is  now  complete. 


Other  variations  of  the  proof  are  possible  with  the  Moi  integral  formula  introduced  in 

equation  (B-6).  For  example,  one  could  start  with  y  =  equation  (B-6)  and 

n  2 

solving  the  resulting  equation  (B-8)  by  polar  coordinates  gives  the  desired  result. 


B-3/B-4 
Reverse  Blank 


APPENDIX  C 

MULTIVARIATE  DISTRIBUTIONS 


For  the  multivariate  case  of  n  independent  random  variables,  the  authors  considered 
functions  of  the  type  y  =  Xj  +  ,  or  the  sum  of  two  random  variables,  Xj  and  x^  <  x  <  co.  *  In 

this  appendix,  the  generalization  of  the  univariate  method  is  outlined.  Reference  4  (chapter  6)  is  a 
good  readable  reference  for  the  material  of  this  appendix. 


NOTATION 

Whereas  the  authors  represented  a  single  random  variable  by  x  and  defined  the  PDFy(^)  in 

^  _  f  \  -  -fix" 

terms  of  the  single  variable  by  the  relation  /  ^  =  and  the  general  Moi 

moments  relation  by  the  function 


nr+- 


-  . 


m  +  J  +  1 


each  random  variable  must  be  indexed  for  clarity.  Let  each  occurrence  of  a  random  variable  x  be 
denoted  now  x  and  let  all  relations  previously  defined  (g'(x),/(x),.£^x^J  etc.)  now  be  subscripted 
appropriately  to  indicate  a  single  occurrence  of  an  independent  random  variable. 

For  example,  let  a  univariate  density  for  a  random  variable  x,  be  denoted 

^  '  r(r.) 

and  let  the  Moi  moments  relation  be  denoted 


r  y^+- 


£(xf)  =  A' 


^1+7  +  1 


m,+\ 

r  — 


The  case  for  a  linear  combination  is  a  straightforward  extension!  Z  =  a,X,  +  CT.X.,  +. .  .+3  X  ,  where  3  .  is  some 

'  11^2  n  n'  j 


assumed  constant. 


where 


m^+l 


If  Xj,  x^,  Xj . x^  are  independent  random  variables  with  Moi  moment  functions, 

E(xl),E(x' ),  E(x'), ....  £«)  and  y  =  .  then 

i 

E(r)=n^(x;), 

1=1 

where 


E(xl)  = 


r,= 


m.+l 

I _ 

n 

I 


(C-l) 


PROOF  OUTLINE 

Because  the  random  variables  Xj,  x^,  x^,  x^  are  independent,  and  thus,  the  joint 

n 

density /(Xj,X2,...,xJ  =  n/(*,  )  by  conventional  definition,  then  the  following  is  true: 

1=1 

00  00  00 

E[y^)  =  J  J  ■  J  -  ’ 

0  0  0 


00 


00 


00 


£(yt)=  \x‘J(x^)ck^, 

0  0  0 


r.+- 

[  1  «i. 

^2+  — 

^  2  ^ 

•/?"n  _ 

X  +  — 

1  ”  «j 

r(/J 


rlyj 


r(rj 


(C-2) 


C-2 


Hence,  the  joint  PDF  is  computed  from  the  following  relation: 


n 

i=l 


I  J  t 

H>'.) 


where 


nj/(x)cfe^.  =1 


(C-3) 


t=l  0 


In  conclusion,  the  case  of  n  independent  variables  involves  taking  certain  products  of  the 
univariate  relations. 


EXAMPLE 

Assume  that  data  are  modeled  by  a  two- variable  function  of  the  form 

Find  the  joint  PDF  and  general  moments  for  ftmctiong.  The 

PDF  is 

f{x^,x^)  =  cg(x^,x^), 
where  c  is  the  normalizing  constant: 

f  '=2  M  v" 


e  ‘ 


J 


(x^,x^)  >  0  , 


so  that 


(C-4) 

C-3 


The  general  moments  function  is  derived  as  follows: 


£(y')  =  fl%) . 

i=l 


'  '  r(i)  r(i) 
£(y^)=6^'/ra) . 


SO  that  the  mean  is  —  and  the  variance  is 
6 


A_  J_-_L 

36  36“  12 


(C-5) 


The  generalization  of  the  multivariate  methods  to  (-00,00)  exponentials  is  a  straightforward 
exercise  and  follows  the  integral  derived  in  reference  4. 
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